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ABSTRACT 

Developing  good  team  skills  usually  involves  exercises  with  all  team  members  playing  their  role.  This 
approach  is  costly  and  has  organizational  and  educational  drawbacks.  For  the  Netherlands  army,  we 
developed  a  more  efficient  and  flexible  approach  by  setting  training  in  virtual  environments,  and  using 
intelligent  software  agents  to  play  the  role  of  team  members.  We  developed  a  general  framework  for 
developing  agents  that,  in  a  controlled  fashion,  execute  the  behavior  that  enables  the  human  player  (i.e., 
trainee)  to  effectively  learn  team  skills.  The  framework  is  tested  by  developing  and  implementing  various 
types  of  team  agen  ts  in  Virtual  Battle  Space  2  (VBS2). 


1.0  INTRODUCTION 

Usually,  team  skills  are  trained  by  exercises  with  a  (potentially  large)  number  of  participants,  each  playing 
a  dedicated  role  in  a  given  scenario.  This  type  of  team  training  requires  ample  resources  (both  human  and 
financial)  and  is  difficult  to  organize.  The  Netherlands  Army  is  in  need  of  new  forms  of  team  training  that 
require  less  organizational  and  logistic  efforts.  A  solution  would  be  to  use  virtual  humans  (agents)  to  play 
the  roles  of  team  members  autonomously.  If  we  can  develop  agents  that  in  training  scenarios  produce 
intelligent  and  realistic  behavior  of  the  individual  that  they  represent,  team  training  can  become  more 
traceable,  more  systematic,  and  more  cost-efficient. 

In  collaboration  with  subject  matter  experts  of  the  Netherlands  Army,  we  approach  this  challenge  by 
developing  a  game-based  platform  for  team  training  in  which  some  team  members  are  played  by  humans, 
and  some  are  played  by  software  agents.  In  this  way,  costs  of  team  training  can  be  reduced,  as  not  all  roles 
have  to  be  fulfilled  by  humans  anymore.  Furthermore,  training  can  be  better  tailored  to  the  specific 
learning  objectives  of  a  specific  individual,  as  we  can  control  more  precisely  the  team  behavior  of  artificial 
team  members  than  that  of  human  team  members  (which  may  be  trainees  themselves). 

We  tested  our  approach  of  agent-based  training  for  hierarchical  teams  and  peer-to-peer  teams.  For  the 
training  of  an  hierarchical  team,  we  used  a  house  search  scenario.  The  commander  is  played  by  a  human 
trainee.  It  is  his  task  to  lead  a  house  search  operation  performed  in  cooperation  with  three  virtual  soldiers. 
The  soldiers  deviate  from  the  original  plans  laid  out  by  the  commander.  They  do  so  because  the  proposed 
hard  approach  does  not  seem  appropriate  to  the  friendly  situation  encountered  in  the  house  (e.g.  innocent 
women  and  children  are  present).  This  requires  the  trainee  commander  to  realize  that  his  original  plans 
seem  no  longer  appropriate,  and  to  take  a  more  comprehensive  approach. 

For  the  training  of  peer-to-peer  teams  we  developed  a  task  in  which  a  group  of  humans  and  a  virtual  team 
member  collaboratively  explore  a  village  to  search  for  weapons  hidden  by  terrorists.  The  team  behavior  of 
the  virtual  team  member  can  be  controlled  on  different  dimensions.  Consequently,  the  difficulty  of 
cooperating  with  the  virtual  team  member  can  be  adapted  to  the  skill-level  of  the  human  trainee. 
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The  remainder  of  this  paper  is  organized  as  follows.  In  Section  2,  we  describe  training  in  hierarchical 
teams.  In  Section  3,  we  describe  team  training  in  peer  to  peer  teams.  The  computational  cognitive  models 
used  in  these  applications  are  described  in  Section  4.  We  report  our  conclusions  in  Section  5. 


2.0  PROVIDING  TRAINING  FOR  HIERARCHICAL  TEAMS 

Hierarchical  teams  are  characterized  by  a  strong  leader  who  is  responsible  for  achieving  the  goals  of  the 
mission.  The  task  of  the  leader  is  to  ensure  that  he  and  his  subordinates  do  the  right  things,  at  the  right 
moments,  in  the  right  way.  Being  a  good  leader  requires  specific  skills,  involving  behaviors  like: 

•  Instructing  subordinates  in  a  clear  and  motivating  way,  taking  into  account  their  different 
personalities  and  attitudes 

•  Planning  and  replanning  in  an  uncertain  and  highly  dynamic  environment 

•  Collecting  the  right  information  from  subordinates  to  make  well-informed  decisions 

To  train  the  capabilities  described  above,  we  used  a  house  search  case.  In  this  case  the  trainee  experiences 
difficult  situations,  and  learns  how  to  solve  problems  under  time  pressure. 

2.1  House  Search  Case 

In  the  house  search  case,  the  commander  (the  trainee)  is  given  orders  by  the  peloton  commander  to  pick  up 
three  potentially  armed  locals  from  their  hiding  place.  The  commander  is  given  intelligence  about  the 
structure  of  the  building,  and  about  potential  risks  that  may  be  encountered.  The  orders  emphasize  to 
refrain  form  actions  that  may  cause  further  damage  to  winning  the  hearts  and  minds  of  the  people,  as  in 
several  recent  incidents  harm  was  inflicted  upon  innocent  civilians.  After  the  briefing  by  the  peloton 
commander,  the  commander  must  instruct  his  men  how  to  perform  the  house  search,  e.g.  following  a  soft- 
knock  or  a  hard-knock  approach.  When  the  commander  enters  the  house  with  his  men,  they  encounter  a 
medium-sized  room  with  three  men  drinking  tea,  and  rifles  lying  on  the  ground.  The  challenge  of  the 
commander  is  to  correctly  assess  the  situation  as  a  friendly  situation  (the  presence  of  rifles  is  not 
necessarily  a  sign  of  hostility  in  that  region),  and  to  ensure  that  his  men  act  accordingly. 


Figure  1 :  A  screenshot  of  the  House  Search  Case  scenario  developed  in  VBS2. 
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3.0  PROVIDING  TRAINING  FOR  PEER  TO  PEER  TEAMS 

A  Peer-to-peer  team  can  be  described  as  a  group  of  people  working  together  on  equal  level  to  achieve  a 
common  goal.  A  good  team  member  maintains  common  ground,  is  mutually  predictable,  and  is  mutually 
directable  [3].  Common  ground  refers  to  knowledge,  beliefs,  and  assumptions  that  enable  the  involved 
parties  to  comprehend  the  messages  and  signals  that  help  coordinate  joint  actions.  Mutual  predictability 
enables  team  members  to  predict  what  the  others  will  do.  This  helps  them  in  planning  and  coordinating 
their  own  actions.  Mutual  directability  refers  to  the  capacity  for  deliberately  modifying  other  parties’ 
actions  in  a  joint  activity. 

Whereas  we  believe  these  properties  to  be  essential  for  team  membership,  how  these  properties  reveal 
themselves  in  behavior  may  vary.  By  explicitly  modeling  such  differences  in  artificial  team  members  we 
can  expose  trainees  to  different  types  of  team  behavior.  For  all  three  aspects  discussed  above,  we  can 
configure  our  agent  in  either  provocative  mode  or  unprovocative  mode.  When  the  agent  maintains 
common  ground  in  an  unprovocative  way,  it  always  shares  relevant  information.  When  it  maintains 
common  ground  in  a  provocative  way,  it  sometimes  refrains  from  sharing  relevant  information.  This 
obliges  the  trainee  to  actively  collect  relevant  information.  When  mutual  predictability  is  implemented  in  a 
unprovocative  way,  the  agent  is  fully  organization  aware;  when  it  is  implemented  in  a  provocative  way, 
the  agent  is  not  (fully)  organization  aware.  This  obliges  the  trainee  to  improve  the  collaboration  with  the 
virtual  team  member  by  communicating  his  or  her  own  roles.  Provocative  mutual  directability  means  that 
the  agent  will  not  unconditionally  obey  a  request  from  the  trainee.  The  trainee  should  then  convince  the 
agent  that  the  requested  action  is  in  the  best  interest  for  the  team  as  a  whole.  In  contrast,  when  the  agent  is 
in  unprovocative  mutual  directable  mode,  it  will  act  upon  any  request  issued  by  the  trainee. 

3.1  Village  Search  Case 

The  scenario  chosen  is  the  standard  NATO  SABRE  experimental  environment  [3].  It  is  a  small  fantasy 
city  with  residential  and  commercial  areas.  This  scenario  was  created  to  investigate  the  differences  in  task- 
approach,  execution  and  performance  in  different  cultures  within  NATO  countries.  In  the  SABRE 
environment  a  team  of  four  members  has  to  find,  identify,  and  neutralize  weapon  crates  hidden  in 
(buildings  of)  the  city.  The  team  has  several  resources  like  scanners,  lock-picking  sets  and  bomb  diffusers 
at  their  disposal  to  accomplish  this  task.  The  number  of  tools  and  the  number  of  times  they  can  be  used  are 
limited.  Team  cooperation  and  coordination  is  imperative  to  accomplish  the  mission  while  disturbing  the 
local  population  as  little  as  possible.  Overall  performance  involve  the  number  of  nuetralied  weapon  crates, 
but  also  the  ‘goodwill’  of  the  population.  An  in-game  measure  of  performance  is  the  team  members’  level 
of  Situational  Awareness  (SA)  at  two  predefined  moments. 
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Figure  2:  The  agent’s  internal  workings  (left);  the  agent  in  the  Sabre  environment  (Right). 


4.0  COMPUTATIONAL  COGNITIVE  MODELS 

This  section  briefly  describes  the  use  of  cognitive  models  for  generating  behavior  of  virtual  characters.  For 
a  more  in-depth  discussion,  the  reader  is  referred  to  [2],  To  specify  the  behavior  needed  for  the  house  -  and 
village  search  scenarios,  we  build  on  Bratman’s  well  known  theory  of  practical  reasoning  [1]  (aka  BDI- 
theory).  At  the  core  of  our  model,  an  agent’s  behavior  is  modeled  by  specifying  beliefs,  goals  and  plans, 
and  by  writing  formal  rules  which  specify  how  beliefs  and  goals  lead  to  action.  To  fine-tune  the  agent  to 
meet  the  demands  of  the  different  scenarios,  we  add  additional  notions  to  our  models.  For  the  house  search 
and  the  village  search  case,  these  are  briefly  described  below. 

In  the  house  search  case,  the  challenge  was  to  program  the  team  members  in  such  a  way  that  they  could 
exhibit  behavior  matching  different  personal  characteristics.  We  have  done  this  by  implementing  goal 
conflicts,  and  different  ways  of  dealing  with  these  conflicts,  depending  on  the  characteristic.  First,  we 
distinguish  the  personal  characteristic  newbie  vs.  leader.  If  a  newbie-agent  encounters  a  goal  conflict,  it 
executes  the  action  (or  goal)  which  it  was  given  by  order  (by  the  commander).  The  leader-agent  on  the 
other  hand,  gives  priority  to  the  goal  it  was  given  by  internal  motivation  (e.g.  stemming  from  prior 
experiences),  and  therefore  sometimes  ignores  orders  of  the  commander.  Second,  we  distinguish  the 
personal  characteristic  peaceful  vs.  trigger  happy  The  peaceful  -agent  prefers  goals  (or  actions)  that  do 
not  lead  to  harming  innocents,  while  the  trigger  happy  -agenthas  the  tendency  to  prioritize  goals  that 
involve  shooting. 

To  implement  the  agents  needed  for  the  village  search  case,  we  focused  on  maintaining  common  ground, 
mutual  predictability,  and  mutual  directablility  (see  Section  3).  In  an  elementary  way,  we  have 
implemented  them  as  follows.  An  agent  maintains  common  ground  with  his  team  members  by  actively 
sharing  relevant  pieces  of  information.  Mutual  predictability  is  achieved  by  applying  an  organizational 
structure  to  the  team.  All  team  members  are  familiar  with  this  structure  and  it  thus  provides  a  shared 
understanding  of  how  the  different  team  members  are  performing  their  share  of  teamwork.  Mutual 
directability  is  implemented  by  a  request  protocol,  which  allows  one  agent  to  ask  another  agent  to  perform 
an  action.  In  provocative  mode,  the  agent  acts  as  an  “imperfect”  team  member.  Provocative  mutual 
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predictability  is  implemented  by  restricting  an  agent’s  access  to  the  organization  model.  The  agent  then 
initially  has  no  knowledge  of  the  roles  of  the  other  team  members  and  therefore  cannot  decide  what 
qualifies  as  relevant  information.  Provocative  common  ground  is  achieved  by  applying  a  random  ‘forget’ 
function  to  the  sharing  of  relevant  information.  As  a  result,  the  team  agent  will  sometimes  fail  to 
communicate  relevant  information.  Provocative  mutual  directability  is  implemented  by  making  the  agent 
attribute  low  priority  to  incoming  requests,  unless  the  request  is  motivated  by  specifying  how  the  request 
can  lead  to  a  fulfillment  of  the  team  goal. 


5.0  CONCLUSION 

In  this  paper,  we  have  described  our  approach  for  developing  virtual  team  members  for  team  training.  We 
have  distinguished  between  agents  designed  in  such  a  fashion  that  they  support  a  trainee  in  learning  how 
to  behave  as  a  leader,  and  agents  designed  to  support  a  trainee  in  learning  to  operate  in  peer-to-peer  teams. 
For  both  applications,  we  believe  that  the  learning  experience  can  be  optimized  by  developing  agents  that 
show  natural  and  realistic  behavior,  and  thus  sometimes  act  imperfectly  or  even  wrongly.  The  Netherlands 
Army  is  in  the  opinion  that  our  approach  to  agent-based  team  training  is  promising,  as  our  models  yield 
behavior  that  is  recognized  as  realistic  for  current  military  operations. 

Our  plans  for  the  immediate  future  involve  using  the  results  and  experiences  reported  in  the  present  paper 
to  develop  effective  and  efficient  team  training  programs.  The  training  value  of  these  programs  will 
subsequently  be  investigated  under  experimentally  controlled  conditions. 
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